Currently pf_get_translation() returns a pointer to a matching
nat/rdr/binat rule, or NULL if no rule was matched or an error occurred
while applying the translation.  That is, we don't distinguish between
errors and the lack of a matching rule.  This, if an error (e.g., a
memory allocation failure or a state conflict) occurs, we simply handle
the packet as if no translation rule was present.  This is not
desireable.

Make pf_get_translation() return the matching rule as an out-param and
instead return a reason code which indicates whether there was no
translation rule, or there was a translation rule and we failed to apply
it, or there was a translation rule and we applied it successfully.

Reviewed by:	kp, allanjude
MFC after:	3 months
Sponsored by:	Klara, Inc.
Sponsored by:	Modirum
Differential Revision:	https://reviews.freebsd.org/D45672

Factor out the bits that run with the sock I/O lock held into a separate
function.  No functional change intended.

Reviewed by:	gallatin, glebius
MFC after:	2 weeks
Sponsored by:	Klara, Inc.
Sponsored by:	Stormshield
Differential Revision:	https://reviews.freebsd.org/D46303

Factor out the bits that run with the sock I/O lock held into a separate
function.  No functional change intended.

Reviewed by:	gallatin, glebius
MFC after:	2 weeks
Sponsored by:	Klara, Inc.
Sponsored by:	Stormshield
Differential Revision:	https://reviews.freebsd.org/D46304

Factor out the bits that run with the sock I/O lock held into a separate
function.  In this implementation, we are doing a bit more work under
the I/O lock than before.  However, lock contention is only a problem
when multiple threads are transmitting on the same socket, which is an
unusual case that is not expected to perform well in any case.

No functional change intended.

Reviewed by:	gallatin, glebius
MFC after:	2 weeks
Sponsored by:	Klara, Inc.
Sponsored by:	Stormshield
Differential Revision:	https://reviews.freebsd.org/D46305

There is no need to hold the sockbuf lock while checking uio_resid.
No functional change intended.

MFC after:	2 weeks
Sponsored by:	Klara, Inc.
Sponsored by:	Stormshield

Just remove the plist created by the respective rule.  Otherwise the two
receipes can race with each other.

Fixes:	d7d5c9e ("pkgbase: Let source packages be built in parallel")
Reviewed by:	bapt, emaste
Reported by:	Mark Millard <marklmi@yahoo.com>
Differential Revision:	https://reviews.freebsd.org/D46320

The pfik_ifp field is not provided by the kernel, it is always NULL. Do not
check for it. This caused us to not clear the skip flag on interfaces, leading
to unexpected behaviour when a 'set skip' was removed.

PR:		280834
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D46311

Reviewed by:	kp
Differential Revision:	https://reviews.freebsd.org/D46312

libgeom maintains a quasi-private mapping of /dev/devstat, which might
grow over time if new devices appear.  When the mapping needs to be
expanded, the old mapping is passed as a hint, but this appears to be
unnecessary.

Simplify and improve things a bit:
- stop passing a hint when remapping,
- don't creat a mapping in geom_stats_open(), as geom_stats_resync() will
  create it for us,
- check for errors from munmap().

Reviewed by:	imp, asomers
Tested by:	asomers
MFC after:	2 weeks
Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D46294

Without this, "make clean ec2ami" won't build a new AMI.

MFC after:	3 days
Sponsored by:	Amazon

8 GB root disk images make FreeBSD/EC2 boot much faster than 6 GB
root disk images.

MFC after:	2 days
Sponsored by:	Amazon

Starting in September 2021 EC2 AMI names have included the boot method:
"BIOS", "UEFI", or "UEFI-PREFERRED".  The third option became available
in June 2023 and allows AMIs to boot via UEFI on EC2 instance types
which support that, but fall back to (much slower) BIOS booting on the
instance types which don't support UEFI.

Since UEFI-PREFERRED is basically a best-of-both-worlds option and is
now the default, there's no point mentioning it in the AMI names.  If
for some reason an AMI is built with the boot method forced to BIOS or
UEFI, that will still be included in the AMI name.

This will not be MFCed, in case anyone has scripts which look at the AMI
names on 13.x/14.x.

Sponsored by:	Amazon

To simplify disabling the kernel sanitizers in some files add
NOSAN_CFLAGS and NOSAN_C variables. These are CFLAGS and NORMAL_C with
the sanitizer flags removed.

While here add MSAN_CFLAGS to simplify keeping KMSAN in kern_kcov.c

Reviewed by:	khng, brooks, imp, markj
Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D45498

To ensure new code has BTI support make it an error to not have the
BTI ELF note when linking the kernel and kernel modules.

Reviewed by:	kib, emaste
Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D45469

Reviewed by:	imp
Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46071

We can share some of the vmm code between VHE and non-VHE modes. To
support this create new files that include the common code and create
macros to name what will be the common functions.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46072

There are some functions that are only needed in non-VHE mode. These
are used to handle hypervisor calls from the kernel, and to manage the
page tables in EL2. As these won't be used by the VHE code we can move
them to the non-VHE specific files.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46073

Teach the virtual timer about the cnthctl_el2 field layout under VHE.
As with non-VHE we need to trap the physical timer and not trap the
virtual timer.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46074

These will become ifuncs to enable VHE in a later change.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46075

Most of the code is identical however some, e.g. managing EL2 memory
or setting EL2 registers, are unneeded under VHE as the kernel is in
EL2 so can manage these directly.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46076

It may cause fields in other registers to change meaning.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46077

To support booting the kernel in EL2 some of the EL0 and EL1 registers
are changed to point to an EL2 version. To get access to the EL0/EL1
version of these registers we need to use the new EL02 and EL12
registers, e.g. to access elr_el1 from the host we would use elr_el12.

Add macros that can be different on VHE vs non-VHE code so these
registers can be accessed in the correct way.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46078

There is no need to store the host par_el1. We don't depend on it not
changing across calls into a guest.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46079

When invalidating the stage 2 TLB we need to ensure page tables updates
have completed and for tlbi vmalle1is the HCR_EL2 TGE flag needs to be
clear.

To fix the former add a data barrier before the tlbi instructions. On
non-VHE this will happen as part of the exception entry, so is only
needed for VHE.

The tlbi vmalle1is instruction operates on the EL2 & 0 regime when
HCR_EL2 E2H and TGE flags are both set. By clearing the TGE flag it
will stop this and operate on the EL1 & 0 regime we are expecting.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46080

When entering into a guest with VHE we need to switch from the kernel
exception vectors to the vmm exception vectors. The latter understands
an exception will be from a guest and can switch back to a kernel
context.

Rather than encoding the location of the kernel vectors we can just
read the value from vbar_el2 and restore it later.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46081

Remove the non-VHE exception code from the VHE code path. As we replace
the exception vectors when entering the guest we don't need to check
which context we are in so can skip parts of the exception vectors.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46082

Follow 1075847 to remove the mention
of -A flag added in b30a7e5 .

Sponsored by:	The FreeBSD Foundation

Allow for local customization.

Reviewed by:	stevek
Differential Revision:	https://reviews.freebsd.org/D46423

Fixes:		70174ef proccontrol: make -s require a target

Stop shipping a log file for etcupdate. This is a source of
non-reproducability as it uses mktemp thereby guaranteeing the output is
different each run.

MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D46317

Add support for specifying how to report the missing Branch Target
Identification (BTI) linker feature on AArch64.

For:

Kernel:    bti-report on when the linker supports it
Userspace: bti-report on when the linker supports it and
           BTI_REPORT_ERROR is defined

Fixes:	43e8849 ("conf: Enable BTI checking in the arm64 kernel")
Pull Request:	freebsd/freebsd-src#1393

(cherry picked from commit 973bbda)

embedfs.S needs the right aarch64 features for BTI and/or PAC.

Fixes:		c2e0d56 ("arm64: Support BTI checking in most of the kernel")

When VHE is enabled we disabled trapping floating-point instructions to
EL2 in locore.S. As the kernel is running in EL2 then vfp.c will manage
enabling floating-point instructions as needed.

Sponsored by:	Arm Ltd

(cherry picked from commit 769eeb0)

Reviewed by:	emaste
Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46513

(cherry picked from commit 7a488d8)

Rename register fields that are only valid when HCR_EL2.E2H == 1. Some
fields move around depending on the value of the E2H field.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46514

(cherry picked from commit 16e6619)

The TTA field moves depending on the HCR_EL2.E2H field. Add a macro to
hold the E2H == 1 case.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46515

(cherry picked from commit 9f3d15f)

Add a new macro that enables all CPTR_EL2 traps. This helps ensure we
trap all extensions we don't support.

Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D46516

(cherry picked from commit d546313)

As with floating point instructions don't trap SVE instructions to the
hypervisor. This lets us handle then in the kernel.

Reviewed by:	imp (earlier version)
Sponsored by:	Arm Ltd
Differential Revision:	https://reviews.freebsd.org/D43303

(cherry picked from commit fe5ed2496e44aec018a6215175bba225b20d81fd)

This was a regression in the merge of the VHE changes which moved the
setting of VBAR_EL2 before CPTR_EL2.

With VHE, the kernel runs in EL2, so defer clearing DDC until locore
has finished using DDC.  In the case of VHE, clear DDC_EL1 to be on
the safe side.